Densities and entropies in cellular automata* 
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, ^ , Abstract. Following work by Hochman and Meyerovitch on multidi- 

^ ' mensional SFT, we give computability-theoretic characterizations of the 

real numbers that can appear as the topological entropies of one-dimensional 
and two-dimensional cellular automata. 
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^ ! Introduction 



Cellular automata are a widely-used model for complex systems or computation, 
consisting in a netvifork of cells each of whose is in one among a finite number 
of states, that is updated synchronously in parallel as a function of the sates 
of its neighbors. Their entropy is a measure of how complex or random the lo- 
' cal long-term behavior can look like. The entropy of cellular automata has been 

. proven uncomputable in [1] (see also [2] for subshifts), but the question remained 

' whether the entropy of a single given cellular automaton could be an uncom- 

. putable number. Recently, M. Hochman and T. Meyerovitch have characterized 

' the entropies of 2-dimensional SFT [3] and 3-dimensional CA [4] as, respectively, 

the right-computable numbers and the limits of computable increasing sequences 
of such numbers. We prove here that these two classes still characterize the pos- 
sible entropies of, respectively, 1-dimensional and 2-dimensional CA. To do so, 
we adapt their homogeneous encoding [3], J. Kari's determinization signals [5] 
. and P. Gacs's self-similar construction [6]. The result brings new equivalences 

between classes that are equally natural in computability theory and dynamical 
systems; we also believe that the construction in itself is promissing, and could 
help understand the real computational power of these natural models. 

In Section 1 we introduce the notions and a brief state of the art. In Section 
2 we state our main results, and the following sections are devoted to sketching 
their proofs. The algorithmic part, as well as entropy proofs and a sketch of the 
main construction can be found in [7] . 
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1 Preliminaries 



1.1 Configurations 

N will denote the set of natural numbers, iNi the set N \ {0} of positive natural 
numbers and the integer interval {i, . . . j}, for < i < j. IR+ is the set of 
nonnegative real numbers. 

Let A be a finite set ealled the alphabet and d G INi the dimension. Any 
element x of is called a configuration, and Xi is called the state of cell i. 
The set of configurations forms a compact topological space when endowed with 
the product of the discrete topology. 

For any q € A, °°g°° denotes the g-uniform configuration of , all of whose 
cells are in state q.lfU G TL'^, x\u is the pattern representing the restriction of 
X to U . For instance, we can define the central pattern a;|B(r) of width r, where 
B{r) = l-r,rt. 



1.2 Symbolic dynamics 

in^ acts on by the shift: to any k & W/^ we associate the homeomorphism 
. ^z<* _^ j^m'' defined by Va; G A^^Vi G 'lL^,u^{x)i = Xi+k- A (d-dimensional, 
or dD) subshift is the set X = (a; G A'^'' C 'Z'',kG TI^,u^{x)\u ^ t\ of 

L finite J 

configurations that avoid some particular set T of finite patterns. Equivalently, 
a subshift is a subset which is invariant by for any k ^1!^ and topologically 
closed. It is of finite type (SFT) if T can be chosen finite. 

Let X C A^** be a subshift. The language of support J7 C Z'' of X is 
£(7(X) = |a; e X}. Its complexity of support U is )Cu{X) — \jC(j{X)\. 

The (topological) entropy of X is %{X) = limr_>.oo "~^"]^7yp^- This is always a 
limit, but may be infinite. Note that if F c B'^'^ is another subshift, then X xY 
can be essentially seen as a subshift of (^4 x B)^ , and its entropy is the sum of 
those of X and of Y. 

A subshift Y C B'^ is a letter factor of X C A'^ if there exists some letter 
projection tt : A ^ B such that the corresponding global map U : X ^ Y, 
defined by the parallel application of tt, is onto (we say that X letter-factors 
onto Y). A subshift is called sofic if it is a letter factor of some SFT. 

The same definitions hold for (one-sided) subshifts over A^^ . 

The trace of X according to vector v <^ and width k is the (d — 
1)D subshift tI(X) = { (a;|[[o,fe[x{o}+nt> )neK| a; e X} over alphabet A^ . The 
directional entropy according to vector v is the limit (X) of the en- 
tropies of T^(X), when k goes to infinity (see [8]). One can see that "K^^ (X) = 
limfe^.oo lim^^-oo l^ij^^^krHl, where A4,r(X) = /C|o,ft|x|o,rl(-'^)- 

Let X and Y be 2D subshifts. We say that X simulates Y with parameters 
B, T if there exists Z d X such that X = Uo<i<B o<i<T (^) ^nd that 
^<BxT> ^ { (a;|[fcB,(fe+i)B|xpT,((+i)T[)/c,(6z| G is a subshift that 

letter- factors onto Y, i.e., any configuration of X can be divided into B x T 



rectangles that project onto letters of Y. A simulation is an r-simulation if the 
letters onto which an array of (2r + 1) horizontally consecutive rectangles of size 
B X T project uniquely determine the central rectangle. 

The following lemma will be useful in the sequel, (ei, 62) denotes the canon- 
ical base for Z^. 

Lemma 1 ([7]). Let X and Y be 2D subshifts such that X I -simulates Y with 
parameters B,T. Then, He^ {X) < U^^ (F) /B and U^^ {X) < U^^ (F) /T. 



1.3 Cellular automata and determinism 

A cellular automaton (CA) is a system F : A^'^ A^'* such that Fa^ = a^F\ 
equivalently there is a radius r € Ni and a local rule / : ^^(''5 A such that 
Va; G Vi G K<^, F{x)i = ). The entropy n{F) of F is the limit, when 

r goes to infinity, of the entropy of the subshift | (-F*(.T)|B(r) )te¥i x G |. 

We say that an SFT X c A^^ is south-deterministic if there is a map 
F : — >■ that maps any line of a valid tiling to a unique line that can 
appear above, i.e., Vx G X,j G = X'Zx{j+i}- It is known that F 

can actually be taken to be the restriction of a CA over alphabet Au{_L}, where 
± must be understood as "extension not defined" ; and the entropy of F is equal 
to "Hei (X) (intuitively, this comes from the fact that state ± will remain forever 
and not contribute to the entropy). X is south-west-deterministic if there 
is the same kind of CA on the diagonal, i.e., Vx G X,j G 7L,F{{xij)i=-j) = 

{Xi,j)i=l-j- 

Let us say that a 2D subshift is SO-sofic if it is a letter-factor of some south- 
deterministic SFT with null entropies, i.e., directional entropy according to 
any vector. 



1.4 Effectiveness 

In ,it is easy to enumerate computationally a base of open sets (consider the 
sets of configurations sharing a given pattern as a central pattern) . That way, we 
can define an effectively closed subset S C as the complement of the union 
of a computable sequence of open sets. It is an effective subshift if, besides, 
it is a subshift. For instance, trace of SFT are effective subshifts. Effectively 
closed sets can also be defined in other Cantor sets; in A^^ they correspond to 
sets of configurations that are not ultimately rejected when scanned by some 
given TM. An effective system is an effectively closed subset S C {A^^)'^ 
which is invariant by the Z'^-shift. Intuitively, it is a dynamical system where 
the preimages of open sets can be computed. 

A III (or right-computable) number is the limit of a decreasing computable 
sequence of rational numbers. A E2 number is the limit of an increasing com- 
putable sequence of 77i numbers. Equivalently, there exists an algorithm that on 
input k outputs the code of another algorithm Mk such that Mk enumerates the 
approximations of a TTi number hk, the sequence hk is increasing and converges 



to h. The set of S2 is strictly larger than the set of ili numbers, which, in turn, 
is strictly larger than the set of computable (Zii) numbers. We refer to [9] for 
more on these classes of numbers (and many more). 

Remark 1. The binary representations of real numbers from an interval [0,a] 
form an effectively closed subset of 2^^ if and only if a is iTi . 

2 Results 

Some evidence of the computing power of a given model can be given by studying 
the class of numbers that can be realized as entropy. Elegant characterizations 
have recently been achieved for multidimensional SFT. 

Theorem 1 ([3,4]). For d > 2, the class of entropies of d-dimensional SFT 
(resp. d-dimensional so fie suhshifts, effective subshifts) is IR+ H 77i. 

In the broader case of eifective systems (and as a consequence for high-dimension 
CA), the class of entropies that can be realized is larger. 

Theorem 2 ([4]). Ford > 3, the class of entropies of d-dimensional CA (resp. 
effective systems) is IR+ Ci S2U {00}. 

The last two theorems have left open the case of entropies realized by ID and 
2D CA, that are both included in IJ2. The main purpose of the present article is 
to solve these two remaining cases. The first step of the answer is given by the 
following result: 

Theorem 3 ([10]). The entropy of a ID CA is equal to the entropy of some 

trace of the corresponding 2D SFT. 

From the theorem above, the entropy of a ID CA is thus iTi. We will actually 
prove that the converse is also true. 

Theorem 4. The class of entropies of ID CA is R+ fl iTi . 

This class of numbers is thus strictly weaker than the possible entropies of 3D 
CA, characterized in [4]. However, this is not true for the 2D case. 

Theorem 5. The class of entropies of 2D CA is R+ fl i72 U {00}. 

3 Construction 
3.1 Density encoding 

This subsection is devoted to encoding data in the density of the configurations. 
The most relevant is actually the binary case, which follows the construction in 
[3]. 

A 1-net is a family (2"Z + kn)ne¥Si of pairwise disjoint subsets of S called 
levels, where {kn)ne'Ni € S^^^. It can be seen that for any 1-net, there is at 
most one cell i e Z which does not belong to any level. 



Let us denote the number of occurrences of letter a in word u. The 
frequency of a letter a € A in some one-dimensional configuration x € is, 
if ever it exists, the limit 5a{x) = limr_>oo |2;|8(r) l^j/l'^l^)!- 

If a,/3 G A^^, we note a ~ /3 if a = /? or there exists i £ Ni such that 
Vj < i,aj = 13 j, and Vj > i,ai = fij and aj = This is an equivalence 
relation, for which all the classes have cardinal one or two. As an example, 
two binary sequences are equivalent for ~ if and only if they represent binary 
expansions of the same real number in [0, 1[. Let A^^ be the quotient of A^^ by 
this equivalence relation. It can be endowed with the induced topology from the 
product topology. We will often confuse a sequence x and its equivalence class. 

If a e A^^ , we note I>„ C the set of Tceplitz configurations which are 
constantly equal to Q!„ on level 2"Z + fc„ for some 1-nct (2"Z + kn)ne¥!- If 
S C A^^, we note Vs = UaeS -^a- These sets have interesting properties. 

Remark 2. 

1. For any nonempty closed set S C A^^, Vs is a nonempty subshift. 

2. The frequency of any letter a € ^ in any configuration x € I^a is =a 2 '- 
In particular if a is binary, then it is a binary expansion of i5i(a). 

3. If Q ~ /3, then = V^; otherwise, Va fl 2?^ = 0. 

4. Let x e Va, j G and i be an odd number. Then x\i'z+j is still in V^- 

Point 3 of the previous remark suggests that it is relevant to talk about V^, (resp. 
Vs) for an equivalence class a € A^^ , or for a real number a G [0,1] (resp. a set 
S c A^^ of classes). 

Moreover, the sequence a encoded in the densities of the subshift can actually 
(up to equivalence) be effectively approximated by reading finite patterns. 

Lemma 2 ([7]). There exists a TM which, given a word u over alphabet 
A, outputs a word v such that, if u = .T||o,2"[ /t"" some x G V^ and some n € N, 
then V = /3|i,„] for some (3 a' and a^^^j = ap^^j . 

We say that a TM has input in A^'^ if it reads sequences of A^^ as input, 
and gives the same result for sequences in the same equivalence class. We can 
also assume that, if a ~ a', then this TM stops after the same number of steps 
for a and a'. 

Lemma 3 ([7]). For any TM M. with input in A^^, there exists a TM M. with 
input in A^^ such that: 

— If Ai halts over input a. G A^'^ , then there exists A: G N such that for any 
configuration x G V^, M. halts over input a;||o,fe| before time k; 

— otherwise, M. does not halt over any input x G V^ ■ 

The following corollary is a direct application of Lemma 3 with a machine 
rejecting configurations outside some eff'ectively closed set. 

Corollary 1. If S C A^^ is effectively closed, then Vs is an effective subshift. 



3.2 Checking homogeneity 



Our proof involves a deterministic SFT that is built layer by layer: the state of 
each cell is in a product of alphabets that we define one after the other, each 
layer having to respect some local constraints in how it can be superimposed 
with the previous ones. For a G A^^ (resp. S C A^^), let us note P* (resp. Pg) 
the set of configurations x C A^ which are constant vertically, and where each 
row is in Vs- for fc e S. 

The purpose of this subsection is to build an SFT which checks that some 
layer is well homogeneous, in the sense of the following lemma; this follows [3, 
Section 6], but contrary to this, keeping determinism and null entropies forces us 
to go back to the actual SFT construction rather than directly invoke Mozes's 
theorem for 2 x 2-substitutions. 

Lemma 4. f^Ni is SO-sofic. 

We will only give a sketch of the proof. A 2-net is a family (/„ x Jn)neTtii of 
products of levels of two 1-nets {In)neNi and (Jn)rieNi- Each J„ x J„ itself is 
called the level n of the net. The In (resp. Jn) being pairwise disjoint, it follows 
that a horizontal (resp. vertical) line can intersect at most one level of the 2-net. 
If i € In, then {i} x 2 is called a column of level n. By definition, columns of 
level n appear with horizontal period 2". 

In [11], Robinson constructed an SFT R in which every configuration is 
divided regularly into squares of size 2" for every n. In particular, he mentions, 
in other terms, the following property about the good repartition of a particular 
state called a cross. 

Lemma 5 ([11]). For every x G R, the set {i G Tl?\ Xi is a cross} is a 2-net. 

Now, this SFT has been made deterministic in [5], by adding to it a layer with 
signals that forbid some configurations that would share the same bottom-left 
half as another one. The result can be restated as follows. 

Lemma 6 ([5]). There exists a south-west-deterministic SFT R that letter- 
factors onto some nonempty subsystem of R. 

Proof (of Lemma 4)- Let us first define a south-west-deterministic SFT R', 

in which configurations are vertically constant and correspond horizontally to 
©^Ni . R' C Rx X is defined with three layers: the first one contains the 

deterministic Robinson SFT the second one is constant horizontally; the third 
one is constant vertically. We additionally require that if the first layer is a cross, 
then the other two must coincide. R' is south-west-deterministic, since all three of 
its layers are. Now it is not difficult to turn this SFT into a south-deterministic 
one, by simply considering R = | (a;(ij_j))(jj)es2 {xij)(^ij)^'z^ S .R'j, whose 

columns correspond to columns of R', but lines correspond to north- west-to- 
south-cast diagonals of R' . 

Null entropies come from the substitutive nature of R, which is transmitted 
to R. More details about this can be found in [7]. □ 



3.3 Checking the density 

In this section, we construct a south-deterministic SFT with null entropies which 
letter-factors onto Vg. In the SFT, there is a special layer which consists exactly 
in Vg-. from Lemma 4, we can a priori assume that all configurations of this 
layers are in 'D^ni ; by implicitly having a layer in R. We will now add a layer 

whose purpose is to check that if a; G I>* is read from this layer, with a G A^^ , 
then a is really in the wanted set S, by simiilating the application of a machine 
M. corresponding to the machine M that rejects any configuration that is not 
in S (see Lemma 3). 

A naive simulation of the machine for an infinite time would create invalid 
limit configurations. A solution to this problem is to build the additional layer 
in a self-similar way, in the fashion of [6,12,13]: we build a family of south- 
deterministic SFT (y„) such that simulates the TM for n steps, and also 
simulates Y^+i with some parameters Bn,Tn. That way, if n was not enough to 
figure out that the input had to be rejected, then a higher level will notice it. 
More precisely, Yn will be able to apply the TM over the input a^|B„_^i|o.B„|+i 
for some j G |0, -Bn+i|. The simulation of Yn+i, as defined previously, consists in 
dividing naturally every valid configuration of Yn into rectangles of size S„ x T„ 
called the F„-macrotiles. An important feature is that this family admits a 
uniform description: one single SFT is actually described. Each configuration 
is conscious of the level Yn it belongs to, and will check that it simulates a 
configuration of the next one. The details of the construction ensuring these 
conditions can be found in [7]. 

The following lemma applies machine A4 from Lemma 3 to finite configu- 
rations composed of some arithmetic progressions in lines of the SFT, that are 
still in Va- Null entropies come from the self-simulation. 

Lemma 7. If S C A^^ is an effectively closed set, then Vg is SO-sofic. 

3.4 Fi"om density to entropy 

Finally, let us sec how Lemma 7 can be used to prove Theorem 4: it simply inde- 
pendently splits each letter 1 into two letters, so that its density is transformed 
into entropy. 

Proof (of Theorem 4)- One direction corresponds to Theorem 3. Let us prove the 
converse. Should we make the product with the shift over 2L"J symbols, whose 
entropy is [aj, we can assume that a G [0, 1[. 

Let F be the shift composed with the CA corresponding to the deterministic 
SFT given by Lemma 7 for the effectively closed set S consisting of binary 
representations of real numbers from the interval [0,a], A its alphabet, and 
TT : A ^ 2 be the corresponding letter projection. Let F be the CA over alphabet 
{A X {0}) U (7r~'^(l) X {1}) such that the first component performs F and the 
second one performs the shift. , i.e., in the first component we can see the 0- 
entropy F and, in the second one the one-dimensional subshift: 

= { (yi)ies e 2^1 3(a;i)iGZ G Ds,Vi G if Xi = 0, then yi = 0}. 



It is known that the entropy of a product is the sum of the entropies, hence the 
entropy of F is that of P J . 

'^uiVj) = E«e£,,(i's)2'"'^ can be bounded by /C[/(Ds)2'"P"^^£/(^s)I''Ii. 
Hence, the entropy 'Hijfg) is: 

i-LyDg ) = lim < tiiPs) + lim sup 



However, since ?^(2?s) = 0, 'H{'Dj) is not more than the maximal density 

a of configurations of Vs- Conversely, if a; G Va C Vs, then D 

{{x^,y^)\^\<r\y^ ^ 7L,yi € {xu2xi}]] hence /CeM(Ps) > 2l"«wli and > 

limsup^^Qo |a;e(r)|i = oi- Therefore, 'W(F) = 'H(X'J) = a. □ 



4 The second dimension 



Let us now prove Theorem 5, dealing with 2D CA. The first inclusion is direct 
from Theorem 2. The idea here will be to realize, in each horizontal slice, some 
right-computable number, as in the previous section. These slices will actually 
be parameterized by some index encoded in its density, that is increased by 
one between consecutive slices, and that will give a sequence approximating the 
wanted S2- The trick is that the encoding has to be sparc^ in order to prevent 
limit configurations to achieve too much entropy; this has to be compensated by 
having actual groups of consecutive slices hold the same parameter. 

Let us denote by {w)i the 4-ary representation of a natural number w G N 
over {0', 1', 2', 3'}. Let [SkjkeMi be a computable sequence of effectively closed 
subsets of , and 5 = { *''{w)iy\ A; e N, < w < 4*= - 1, y e J U {*°°, ti°°} 
a set of sequences over alphabet A = {*, 0', 1', 2', 3', 0, 1, jt}. Consider 5' = U 
5*2 U 5*3, where: 

Si = { (z, z') e 52 1 3fc, y, m; G |0, 4*= - 1 1 , z = *\wUy and z' = + 1)4^} ; 

5*2 = { (z, z') G 52 1 3k, y,z = *'=(4'= - 1)4?; and z' = ti°°} ; 

5-3 = {{z,z') eS^\z = *°° and z' = or 3k,y,z' = *''{0)4y}- 

Lemma 8 ([7]). S' is an effectively closed subset of {A x A)'^!. 

We are now ready to characterize the entropies of 2D CA. Similarly to the 

one-dimensional case, a 2D CA corresponds to a south-deterministic 3D SFT, 
up to adding a spreading state, and its entropy can be seen as the directional 
one for the south-to-north unitary vector. 

Proof (of Theorem 5). Let ak be a computable sequence of 77i numbers, Sk = 
[0, ak], M the TM given by Lemma 8, Y the 2D SFT given by Lemma 7. 

Consider now the following 3D SFT Y': each horizontal slice must satisfy the 
conditions of Y. The only vertical local constraint we add is the following: the 



second letter (in A) of the pair held by a tile is equal to the first letter of the tile 
on top of it. Intuitively, the way to think about this is that when a horizontal 
slice is considering whether it should accept or reject its input (the first sequence 
it holds) , it can also read as input the sequence of the slice above it (the second 

sequence) . 

Y' is south-deterministic. Indeed, every horizontal slice is an element of Y, 
which is a 2D south-deterministic SFT. Hence, if we know a slice a;|2x{n}xZ) 
we can uniquely determine •^;|zx{ri+i}xz • Moreover, Y' has null entropies, as a 
subshift of an infinite product of 2D SFT with null entropies. 

Let us now modify the SFT in order to get the wanted entropy. We need 
to understand the structure of the configurations. From now on, wc forget the 
second sequence encoded in every horizontal slice and we work only with the first 
one. If Zk is the sequence encoded in the fcth horizontal slice, then the sequence 
{zk)ke'z can only have one of the following forms: 

- Zfe = for all fc e S; 

- there exist m e S,fc G Ni and y G T^[o,ak] such that Zi = *°° for i < m, 
Zi = *'^(i — m)j^y for m < i < m -|- 4*^, and Zi ='^°° lov i>m + 4^^. 

- Zk = t)°°, for all fc e 

This follows directly from the definition of S' . For k G Ni, let Y'{k) C Y' consist 
of those configurations whose horizontal slices are either *°°, tj°°, or contain 
*'^(0)4j/ for some y G ^^fo.Qfc]- It is a subshift. 

Let us allow splitting of the letter 1 into two (by adding a second, binary, 
layer, as in the proof of Theorem 4), independently in every horizontal slice. 
Then, in configurations of the subsystem Y'{k) there are 4*^ slices where splitting 
is done and each one contributes up to 4~'^/3 to the entropy, where /3 G [0,afc] 
is such that y G /3. This happens because in every slice, y is encoded in 2-net 
starting from level 2k. Since splitting is done independently in 4^^ slices, the 
entropy of the subsystem Y'{k) is /3. By the variational principle, and since the 
nonwandering system of the CA is included in the disjoint union of the Y' (k) and 
the trivial subsystems, we have that the entropy of F in the vertical direction is: 

T-LiF) = sup sup j3 = sup ak, 
which is the wanted S2 number. □ 

Conclusion 

We have reached a characterization of the entropies of CA in terms of com- 
putability classes. This is inspired by what had been done over multidimensional 
SFT, but the construction presents some intrinsically interesting points, such as 
determinization widgets, self-similar construction, or a generalized encoding of 
configurations into densities. 

This problem helps us understand what kind of results on tilings could be 
adapted to CA, that is when one of the dimensions of the system actually repre- 
sents a deterministic temporal evolution. It could be interesting to try to adapt 



some more results from multidimensional symbolic dynamics, such as the sub- 
stitutions of [14], or the characterization of subactions in [4, 12]. Nevertheless, 
when translating into cellular automata, we will in general have to deal with 
wandering points, which could be omitted here in the study of entropy but may 
sometimes alter significantly the results. 

Among open problems, we could try to characterize the entropies of restricted 
classes of CA: requiring transitivity constraints, or reversibility. The latter case 
might be achieved by adapting our proof while requiring two-way determinism in 
the underlying tilings (but again extending it to a full set of configurations may 
be difficult). We could also study the entropies of other computationally- inspired 
dynamical systems, such as Turing machines with moving tapes. 
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A Algorithms 



Proof (of Lemma 2) . Consider the following algorithm: 
Require: A word u = ui . . . over alphabet A. 

if \u\ = 1 then 

return the empty word. 

end if 

if 3a G ^, 7 C , |/| = \u\ /2, yi&I,Ui = a then 

return a concatenated to the word returned by this same algorithm applied 

to {Ui)i^i. 

end if 

return Error. 

Let (fcj)jeNi and a £ A^^ be such that Xi = aj for any i G 2^'E + kj and any j e 
Ni. The definition of 1-net gives, for any j < n, ||0, 2"| n (2-'Z + kj)\ = 2"--''. 



The levels being disjoint, we get 



[0, 2"[ \ U,<„(2^'^ + kj) = 2" - E.<„ 2" 



1. Besides, the number of occurrences of any letter a € Am u 



a;||o,2'> 



IS 



u\ = 



E 

j'^n,a.j —a 



|[o,2"[n(2^z + fc,-)| 



i e 



[0,2l\ |J(2^Z + fc, 



j<n 



which is equal, with a difference of at most 1, to V . , 2" ^ . In particular, 

ai occurs at least 2""-^ times. If the algorithm choses 7(m)i = ai, then the 
statement is obtained by recurrence on the logarithm of the length of u. Now 
if the algorithm choses 7(^)1 7^ ai, it means that these two letters were both 
equally present (each covering half of |0, 2"|). It is not difficult to see that this is 
possible only if aj = 7(w)i for any j G |2, n], and that in this case the algorithm 
will output 7(u)ia"~"^ = Pii^nj for /3 = 7(u)iai°° ~ ai^{u)i°°. □ 

Proof (of Lemma 3). Let M. be the TM performing the following algorithm: 



Require: An infinite word x G A^^ . 
for t e Ni do 

Apply A^|> (from Lemma 2) to X||o.2*[[ : let 1; its output. 
Perform t steps of algorithm Ai over finite input v. 
end for 

— Assume that M. halts over input a G A^^ after some time t. Let n be the 
time nc;cd(;d to perform completely the loop t of the algorithm (over some 
infinite input). From the algorithms of M and Alt>, it can be seen that this 
time depends only on t (and M., but not on the input). Let x € be some 
configuration. At loop i we have computed v = /3|i,t] for some (3 ^ a' and 
ajjj^ q = Q:|x.f|, and simulated A4 over input /3. The machine halts before t 
steps over input a, so it also does over input a', since they have the same 
prefix of size t, so it cannot make the distinction between the two at that 



point. Prom the property of the machine and the fact that /? 
conclude that it also stops in t steps over input l3. 
— On the contrary, assume that M halts over some input x e This means 
that M halts within t G Ni steps over some input v, which was computed 
by A1>, which is then equal to some v = /3|i,t] for some /3 ^ a' and a'j^ = 

Q!|i,t|. Then it means that A4 halts over input a' ^ /3 before t steps, hence 
over input a. □ 

Lemma 9. S is an effectively closed subset of A^^. 

Proof. We can see that S respects the equivalence classes. The following algo- 
rithm effectively rejects exactly the elements outside S: 

Require: A configuration z = Z1Z2 ■ ■ ■ S A^'-. 
if 2:1 = ji then 
for fc G ]Ni do 
if Zk 'i then 

return Error, 
end if 
end for 
else 

for fc G Ni do 

if Zk *, break out of the loop, and remember fc. 
end for 

for i G |fc, 2fc - 2] do 

if Zi ^ {0',1',2',3'} then 
return Error. 

end if 
end for 

Apply the TM corresponding to Sk to the sequence Z2k+iZ2k+2 ■ ■ ■ 
(it should be considered as rejected if some letter Z2k+i ^ 2). 
end if □ 

Proof (of Lemma 8). It is easy to design an algorithm that rejects all the pairs 
of words outside of S' using the algorithm of Lemma 9 as a subroutine. □ 



B Null entropies constructions 

Proof (of Lemma 1). We prove the claim only for the vertical direction, since the 

horizontal case is analogous. By definition, "^£2 {^) = limfe->oo limr_>oo '"^'^^'''^^^ 
By passing to a subsequence, wc can write fc = k' B and r = r'T. Then, a 
rectangular pattern b of size fc x r is contained into a pattern consisting of 
(fc' + 1) X (r' + 1) rectangles of size B x T, each of which corresponding to a 
letter of Y. Since the simulation has radius I, if wc thicken this pattern on the 
left and right by / to obtain a (fc' + 1 + 21) x (r' + 1) pattern of Y, then the 
central (fc' + 1) x (r' + 1) S x T-rectangles are uniquely determined. Finally, this 



pattern together with the coordinates of the bottom-left corner of b in the BxT 
rectangle that contains it uniquely determine b. Therefore, 



yX) = hm hm 

< lim hm = 

fe'->-cx> r'— >oo r'T T 

which proves the claim. □ 

Proof ( of the entropy part of Lemma It remains to prove that R' has null en- 
tropies. First of all, we prove that T-Lf,^ (R) = 0. This is true because there exists 
an SFT N which letter-factors onto R and 0-simulates itself injectively with pa- 
rameters 2,2 by [15]. Therefore, according to Lemma 1, ?/e2 {^) ^ {^) /2, 
which means that He^ i^) = 0. Since N letter-factors onto R, we also have that 

^e2 {R) = 0. 

Adding the diagonal signals of ^ does not increase the directional entropy: it 
is mentioned in [5] that for rectangles of arbitrary size, there are only 4 cells where 
we have a choice for the diagonal signals. Therefore, A/fe.r (^) < s^J^k.r (R) ■, where 
s is the number of diagonal signals. From the last equation, we immediately get 

that He^ (r^ < -Hea {R)- 

Similarly, the horizontal and vertical signals of R' do not increase the di- 
rectional entropy. This is true because a square of size k x r intersects at most 
(log A; + 1) + (logr + 1) different levels of the 2-net. If we specify the letters of 
these levels, then all of the signals are uniquely determined. There are 2 choices 
for every level, therefore A4,r(^') < 2^°^''+^°s''+'^J\fk,r(^) < AkrJ\fk,r- Hence, 

In the same way, we can prove that 'Hei = 0. By [16], this implies that 

all the directional entropies of R! are equal to 0. Since i? is a shifted version of 
it' , for every v G 71?, 'Hv (jij = 0, which is what we wanted to prove. □ 

Lemma 10. x G D* is represented in Yi if and only if a ^ S. 

Proof - Let y E Yi be such that the 2?^xi layer of y is .t G 7?* . Let x G T>a de- 
note a line of x (recall that x is constant vertically) . y„-macrotiles present in 
y have applied, without halting, n steps of TM M over input i|B„+i|o,B„[+j 
for some j G |0, From Point 4 of the remark in Subsection 3.1, 

x^Brt+i'z+j G Va. Thus we have, for any n, a configuration of T>a that is 
not rejected by M within n steps. By Lemma 3, this means that M. does 
not halt over input a, i.e., a E S. 
— Assume, on the contrary, that Ai does not halt over a. By Lemma 3, Ai never 
halts on any configuration a; G Va (and any ); hence there exist 

y„ macrotiles for every n. Every y„ macrotilc gives rise to a B1B2 ■ ■ ■ Bn x 
TiT2---Tn rectangle validly tiled by Yi. Therefore, Yi can tile arbitrarily 
large rectangles, which means that it can also tile the plane. □ 



Proof (of Lemma 7).Yn, from Lemma 10, is a south-deterministic SFT which 
letter- factors onto 2?^. 

Let us now prove that 'He2 {^n) = (a symmetrical argument will give that 
"Hei {Yn) = 0, and [16] has us obtain all directions). For every n S Ni, Yn 
1-simulates l^+i with parameters B„,T„. On a second layer, there is a config- 
uration of R, which is independent from the first 1^-layer. Therefore, Lemma 1 
gives that (y„) < + [r^ = 2l£ai^i±ll. Inductively, we can 

show that for every n, m e Wi, "Hea (^n) < '^'^y^^-'^y-n+i) _ Also, Tn+m > -B„+m > 
log |A„+m+i|, where An+m+i is the alphabet of the SFT This implies 

that "Hes (i^n+m+l) /Tn+m < 1, hcnCC (Yn) < j;,- -Tl+m-l ' ^^^^'^ ^ 

Since r„ > 1 for all n, it follows that H^^ {Yn) = 0. □ 



C Details of the macrotile construction 
C.l Fields of the macrotiles 

Each y^-tile contains a state divided into the following fields: 

Level: This is equal to the unary word 1". 
Addr, Age: These two fields contain two integers i £ |0, and j € |0, respectively, 
that correspond to the coordinates of the tile. The local constraint for the 
coordinates arc quite natural: the right neighbor of a tile with coordinates 
{x, y) must have coordinates (x -|- 1 mod Bn, y) and its upper neighbor must 
have coordinates {x,y + 1 mod T„). 
Info: This field contains a letter from the alphabet F = {0,1, /,(!}• For every 
y„-macrotile, the word over F of length Bn consisting of the Info fields of 
the tiles at positions (i, 0)o<i<B„, represents the complete description of the 
y„+i-tile that it simulates. We can now refer to this word as the state of the 
y„+i-tile. 

The letters and 1 are used for the binary encoding, / is used to separate 
different subfields of the simulated tile, and (1 is used as an endmarker. The 
y„+i-tiles have the same structure as the 1^-tiles. Hence, the simulated tile 
will be divided into subwords separated by the / symbol, and each subword 
will contain the information of a field of the simulated tile, i.e., the simulated 
tile will have the form Inf o.Level/inf o.Addr/ . . . /inf cCheckjltt . . . jj (again 
keeping in mind that the fields Inf o.Addr, Inf o. Age, etc. cannot be read from 
a single Y^-tile, but rather they are written letter by letter on a segment of 
tiles). 

At this point, note that we must have log |5^+i| < Bn in order for the states 
of y„+i to be represented with words of length 
Lmail, Rmail: These fields will be used to exchange information between neighboring y„- 
macrotiles. They have the same alphabet F as the Info field. The Lmail 
field will send the information of the simulated tile to the left-neighboring 
y„-macrotile and the Rmail field to the right neighbor. In this way, every 
macrotile will learn the simulated tile of its neighboring macrotiles. 



Prog: This field contains the description, encoded over alphabet {0, 1}, of an al- 
gorithm that, given as inpiit n and the states of three 1^-tiles, outputs the 
unique tile that can be placed above them in F„ (or rejects if there is none). 
At this point lies the heart of the construction: the program written in the 
Prog field is a uniform program that governs the behavior of all y„. 

Work: This field will be used to store intermediate data during the computation. 
Check: This field will be used as an input for the TM simulation by the macrotile. 
It is devoted to containing the character that is present in the layer of ^^^nj 
at the bottom-left corner of the macrotile. For n = 1 we require that the 
Check field of a tile is equal to the letter in the layer I'^nj ■ 

As we have already pointed out, any configuration of Yi will have a very 
strong hierarchical structure: it can be divided into Fi-macrotiles, i.e., rectangles 
of size Bi X Ti with coordinates (0,0) on the lower left corner, that behave like 
F2-tiles, and the local constraints of Y2 are satisfied between Fi-macrotiles. In 
particular, rectangles of size B1B2 x T1T2 of Yi-tiles will form y2-niacrotiles 
which simulate Y3 tiles. And so on: for every n, every valid configuration of Yi 
can be partitioned into Yn macrotiles of size Bi ■ ■ ■ Bn x Ti • • • T„ that simulate 
y„+i-tiles. 

C.2 Self-simulation 

In this subsection, we will give further details about the construction of F„, and 
we will also describe how it simulates Fn+i- The worktime of a i^-macrotile 
is divided in various subperiods during which an agent will perform different 
operations. These phases are the following: 

1. Sending mail: During this period, the Y^-macrotiles exchange their informa- 
tion, so that a macrotile learns the state of its left and right macrotile. To 
do this, we assume that at position (0, 0) there is an agent (a TM head that 
organizes the computation) that performs the following operation. It starts 
moving to the right until it meets the marker (J in the Info field. Then, it 
turns to the left and starts reading the word backwards, at each step copying 
the letter that it reads onto the Lmail field, in which they move one step to 
the right at every time step. In this way, we will have a caravan of letters 
separated by one cell moving to the right in the Lmail field. It will take them 
Bn steps to reach the right-neighboring macrotile. At the same time, when 
the agent reaches position 0, it starts walking back to the right again until it 
reaches again the first (t in the Info field. Since the length of the word in the 
Info field is of length 0(logB„), the agent has enough time to reach this 
position before the Lmail from the neighboring macrotile starts coming in. 
When the first letter arrives, the agent stops it and moves one position to 
the left. Then, it stops the next letter that it meets and so on, until it even- 
tually reaches position 0. In this way, the Info field of the left- neighboring 
macrotile has been copied onto the Lmail field of the macrotile. This whole 
procedure has taken up time B„ -|- 0(logS„) < 2S„. After that, the agent 



can organize a similar procedure to copy the mail of the right neighbor of a 
macrotile to the Rmail field. This takes another 0(i3„) steps. 
An important remark is that the whole process does not depend on n. 
Namely, we can describe the action of the agent and of the letters mov- 
ing in the Lmail and Rmail fields with a uniform program that works for 
every value of B„. 

2. Checking the level: In this workperiod, we make sure that the Info. Level is 
greater by 1 than the Level of the tiles that form the macrotile, in order to 
ensure that the F„-macrotile simulates a y,i+i-tile. 

The way to achieve this is the following: our agent calculates the numbers 
in Addr (represented in binary) and in Level (represented in unary). Then 
it checks that Info is 1 if Addr < Level + 1, and / if Addr = Level + 2 
(remember that Info. Level is written first on the Info field). This whole 
comparison can be done by a TM independent from i3„ , T„ in time and 
space 0{logBn + n) = 0(logi3„), assuming B„ >> n. Also, at the end 
of this workperiod, we can assume that the agent returns to the cell with 
address 0. 

3. Checking the coordinates: Now, every macrotile is conscious of the Info of 
its neighboring macrotiles. Recall that the word read in the Info fields of a 
macrotile has the form Info. Level/Info. Addr/Info. Age/ . . . /Inf o.CheckjJjJ . . . jj. 
The Inf o. Addr and Inf o. Age fields are binary representations of coordinates 

i € |0, and j £ [0,r„+i| respectively. These words are of length 

log-B„+i, logT„+i = 0{Bn). We will say that the address of a macrotile is i 
when the macrotile has a binary representation of i in its Inf o. Addr field. 
In the beginning of this workperiod, the agent reads the word 1" written into 
the Level field of the y„-macrotilc. Then the agent writes down the binary 
representation of .Bn+i and Tn+i in the Work field. A new restriction on how 
to chose Bn and T„ (see Subsection C.3) is that, given 1" as input, we can 
write down a binary representation of and Tn+i in time polynomial in 

n. 

After that, the agent goes to the cells that encode Info. Addr (it can be 
found by reading Info until finding a /) and checks whether Info. Addr = 
Lmail. Addr + 1 mod Bn+i- If they are not, then the tiling becomes invalid. 
This check can be done in time polynomial in logB„ and space 0(i3„). 
Similarly, the agent also checks that Inf o. Age = Lmail. Age. 
At the end of this worktime period, we have assured that the coordinates of 
the macrotiles are compatible. The operation in this level depends on n, but 
in a uniform way, i.e., there exists a fixed TM that, given n, outputs B„ and 
T„. This input can be read from Level field. 

4. Transmitting the Check field: During this workperiod, we want to make sure 
that the information of the Check field corresponds to the data in the layer 
X>^Ni . A local constraint already imposed this for the Yi -macrotiles. The 
other levels cannot perform the same (in order for the description to be 
uniform), but instead, the level can be transmitted between the levels. To 
do so, the agent goes to the cell with address 0, memorizes the bit it sees in 
the Check field and then checks that the Inf o. Check field is equal to this bit. 



This can all again be done in time 0(log This process can be performed 
by a program independent of n. 

The result of this process is that a Y^-macrotile supported on the rectangle 
lx,x + Bnfx ly,y + Tnl carries in its Check field the character from the cell 
{x, y) of the layer X'^xi . Wc assume that i?„ is odd, so that (see the Point 4 
of the remark in Subsection 3.1) the Check fields of the configuration of 
and the configuration of l^+i form two configurations x,y € for some 
a e A^^ (they have the same densities). 

5. Checking the input: We can assume that at the beginning of this workpcriod, 
the agent is in the cell with address 0. During this workperiod, the agent 
performs n steps of the TM M., with input tape the Check field, and working 
tape the Work field. If halts within n steps, then the configuration is 
rejected (it is not in our final subshift). 

6. Forcing self- similarity: In this workperiod, we ensure that Y^-macrotiles be- 
have like y„+i-tiles. The behavior of 1^-macrotiles is governed by the pro- 
gram written in Info. Prog, so we have to check that it is the same as the 
program written in Prog of every tile of Y^. This is easily checked in the fol- 
lowing way: the agent goes to the cell where the first letter of the Info. Prog 
is held. Again, this can be done since the encoding of the information of the 
simulated tile in a macrotile is algorithmic and has a very explicit form. In 
this cell, it checks that the letter held in the Info field is equal to the first 
letter of the Prog field. Then, the agent goes one cell to the right and com- 
pares the letter in the Info field with the second letter of the Prog field, and 
so on, until a / is read in the Info field. If at any cell the letters examined 
are not equal, the configuration is rejected. This period takes 0(log _B„_|_i) 
steps. 

7. Updating the state: The agent can apply the local rule encoded in Prog 
to the states described in the Lmail Info and Rmail fields. It does so by 
comparing this triple to the ones encoded in Prog. This will eventually give 
the result, which is written in the Info field bit by bit. Then, the agent clears 
the Lmail, Rmail and Work fields. Fields Level, Addr, Prog and Check are 
left unchanged; field Info. Age is incremented by 1. This period also takes 
0(logi?„+i) steps. 

C.3 Choosing the right values for Bn and Tn 

In the previous construction, we have made some assumptions concerning the 
values of S„ and T„. First of all, let us notice that for every n, if An is the alpha- 
bet of y„, we have \An\ < O(nBnTn)- Also, the whole computation contained 
in a y„-macrotile can be done in time 0{Bn), hence Tn can be chosen to be of 
order 0{Bn). Bn must satisfy the following restrictions: 

— log|^|„<B„; 

— Binary representations of Bn and T„ are computable in time polynomial in 
n with input given in unary; 

— Bn is odd; 



- Bn » n. 

The first restriction is necessary in order to be able to represent 1^+i-tiles with 
y^-macrotiles. The second one so that T„ can be chosen relatively small com- 
pared to Bn- The third one is necessary so that we can read a from the sequence 
of macrotiles. 

If we choose B„ = ci3" and T„ = C2-B„, where ci, C2 are fixed numbers and 
Ci is sufficiently large so that the inequalities are satisfied for small values of n, 
then all of the restrictions are satisfied. 



